1. Technical Field
The present invention relates to an image processing apparatus and an image processing method and, particularly to an image correction technology.
2. Background Art
Recently a technology of correcting a tilt of an image has been proposed in a camera shake correction technology in an imaging apparatus such as a camera and a movie camera.
In a method disclosed in PTL 1, an acceleration sensor is provided in the imaging apparatus, the acceleration sensor measures gravitational acceleration to detect the tilt of the imaging apparatus relative to an earth axis, and the tilt of the image captured based on a detected tilt angle is corrected. In a method disclosed in PTL 2, a line segment existing in an upper portion of a captured image is detected, the tilt angle of the whole image is estimated from the tilt angle of the line segment, and the tilt of the captured image is corrected. In a technique disclosed in PTL 3, a sensor and image processing are used while combined, thereby improving accuracy of calculation of the tilt angle.
In a method disclosed in PTL 4, when the tilt angle of the image is calculated using the image processing, the image is divided into small blocks, directionality indicated by a texture in each block is determined, and only a domain in which the texture has a unidirectional characteristic is targeted to calculate the tilt angle. The method disclosed in PTL 4 is equivalent to extraction of the tilt information only from a structure in the image, and contributes to the improvements of accuracy and stability of the calculation of the tilt angle.
However, in the method disclosed in PTL 1 in which the sensor is used, because an output value of the sensor includes a fluctuation component such as an inertial noise or sensitivity of another axis, the tilt is hardly corrected with high accuracy when a user captures the image with the movie camera while walking. In PTL 2, there is a restriction to a composition of the captured image. Therefore, the method disclosed in PTL 2 is impractical for general usage.
PTL 3 and PTL 4 propose that the sensor and the image processing are used while combined in order to compensate the above defect. That is, there is proposed the method for selecting an angle component satisfying a certain standard from a plurality of candidates of the tilt angles using the sensor information. However, the accuracy of the output value of the sensor is inevitably degraded when the fluctuation component overlaps the output value of the sensor. Additionally, PTL 4 also discloses the method for improving the accuracy of the estimation of the tilt angle by previously removing tilt angle information from a non-structure that possibly becomes the fluctuation component during the estimation of the tilt angle. However, PTL 4 cannot exhibit the effect when information on the tilt angle relative to the structure becomes a fluctuation component as it is.
FIGS. 1A and 1B are views for explaining examples in which the tilt angle information from the structure becomes the fluctuation component. FIG. 1A illustrates the image captured with a wide-angle lens such as a fisheye lens. Generally, because a distortion exists in the captured wide-angle image, in order to obtain a correct result of the image processing, it is necessary to cut out a partial domain from the captured wide-angle image to produce the image having the small distortion by a calibration or a back calculation from a lens projection method. FIG. 1B illustrates an example of the image in which a neighborhood of a center of FIG. 1A is cut out to correct the distortion. As can be seen from the compositions of FIGS. 1A and 1B, a horizontal line of the structure extends from the front toward a depth direction, and the horizontal lines converge toward a disappearance point while being originally parallel lines. The number of vertical lines of the structure, which provides the correct tilt angle information, is relatively decreased compared with the horizontal line. Therefore, the correct tilt angle information is hardly detected. FIG. 2 illustrates an angle histogram of luminance gradient in each pixel of the image of FIG. 1B. In FIG. 2, a horizontal axis indicates an angle, and a vertical axis indicates a frequency. Assuming that the image is tilted by an angle θ, normally it is desirable that a frequency of bin of θ really takes a mode value. However, as can be seen from FIG. 2, a frequency of bin of the angle obtained from the horizontal line of the structure exceeds the frequency of bin of θ. Because the characteristic becomes particularly prominent by the wide-angle image, the characteristic cannot be solved by the method disclosed in PTL 4 in which the non-structure is previously removed.
In a method disclosed in PTL 5, a candidate of the disappearance point and an edge extracted from the image are connected by the line segment to produce a histogram in which the frequency of the tilt angle of the line segment satisfying a predetermined standard is accumulated, the histogram is obtained while the candidate of the disappearance point is continuously changed, and the candidate of the disappearance point indicating the largest frequency is determined as the disappearance point. When the disappearance point is determined, the horizontal line of the structure can be specified to remove the fluctuation component from the structure. However, in the method disclosed in PTL 5, there is also a possibility of removing the vertical line of the structure. It is also necessary to previously fix the predetermined standard, and the method of PTL 5 cannot be used when the tilt angle is unknown in capturing the image.
An object of the present invention is to enable the tilt angle to be estimated by selecting information that can be used to estimate the tilt angle of the image from an edge component obtained from the image and the tilt angle of the edge component and to correct the tilt of the image.